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Apparatus, Method and Computer Program Product for 
Guaranteed Content Delivery Incorporating Putting a Client 
On-Hold Based on Response Time 

Background of the Invention 

Field of the Invention 

The present invention relates generally to Internet client-server 
applications, and more specifically to guaranteeing network client-server response 
time while providing a way of putting the client on-hold when the response time 
temporarily prohibits access to the requested server. 

Related Art 

The importance to the modem economy of rapid information and data 
exchange cannot be understated. This explains the exponentially increasing 
popularity of the Internet. The Internet is a world-wide set of interconnected 
computer networks that can be used to access a growing amount and variety of 
information electronically. 

One method of accessing information on the Internet is known as the 
World Wide Web (www, or the "web"). The web is a distributed, hypermedia 
system and functions as a client-server based information presentation system. 
Information that is intended to be accessible over the web is stored in the form of 
"pages" on general-purpose computers known as "servers." Computer users can 
access a web (or HTML) page using general-purpose computers, referred to as 
"clients," by specifying the uniform resource locator (URL) of the page. FIG. 1 
is a network block diagram showing a plurality of clients and servers connected 
to the Internet 

When a client specifies a URL, a part of the URL known as the Domain 
Name is passed to a domain server (DNS) to be translated to a network address. 
The network address specifies the Internet protocol (IP) address of the intended 
or requested server. The client request is passed to the requested server having 
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the network address. This process of how the client request eventually arrives at 
the requested server is operating in virtual IP mode. In this mode, the address of 
the requested server is never published and thus not accessible to the client. Here, 
the "server uses the path name in the URL to locate the web page requested by the 
client. A copy of the web page is then sent to the client for viewing by the user. 
A second mode, called standard mode, will be briefly described next. 

In standard mode, the actual address of the requested server is published. 
Here, the client's specified URL does not include the Domain Name. Therefore, 
the request does not go through the DNS to eventually arrive at the requested 
server. 

The client-server paradigm described above has served the Internet well. 
However, there are some problems. One problem is unacceptable server response 
time when too many clients are requesting a page from the same server. In fact, 
when this number of clients (i.e., users) exceeds a certain threshold, often times 
the result is a server failure or crash. A computer crash means that the computer 
itself stops working or that a program aborts unexpectedly. A crash signifies either 
a hardware malfunction or a very serious software bug. When the response time 
for a server is unacceptably slow and/or has a tendency to crash often, the owner 
of the server may lose business. This loss of business is detrimental to a business 
person who mainly conducts his or her business on the Internet. 

Increasing server response time to a client request may be the result of 
several factors. One such factor is multitasking. Multitasking, which is well 
known in the relevant art(s), is the ability to execute more than one task (e.g., URL 
or page request) at the same time. In multitasking, only one central processing unit 
(CPU) is involved, but it switches from one task to another so quickly that it 
attempts to give the appearance of executing all of the tasks at the same time. 
There are two basic types of multitasking: preemptive and cooperative. In 
preemptive multitasking, the operating system parcels out CPU time slices to each 
task. In cooperative multitasking, each task can control the CPU for as long as it 
needs it. If a task is not using the CPU, however, it can allow another task to use 
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ittemporarily. OS/2, Windows 95, Windows NT, the Amiga operating system and 
UNIX use preemptive multitasking, whereas Microsoft Windows 3.x and the 
MultiFinder (for Macintosh computers) use cooperative multitasking. 

Whether the operating system of a particular server uses preemptive or 
cooperative multitasking, the response time to URL (page) requests increases as 
the number of requests increases. Under certain load conditions, these page 
requests can represent a considerable overhead, consuming a large percentage of 
the server's processing resources. What is needed is a way of guaranteeing the 
response time to a client's page request at a particular server (or web site), and 
thereby avoid wasting valuable server CUP time and crashes. In addition, what is 
needed is a way of providing alternative information to the user (via the client) 
when the server containing the desired page is not available. By providing 
alternative information to the user it helps to not only generate possible related 
business but also redirects the user back to the desired page once the server is free. 

Summary of the Invention 

The present invention is an apparatus, method and computer program 
product for guaranteeing network client-server response time while providing a 
way of putting the client on-hold when the response time temporarily prohibits 
access to the requested server. In a preferred embodiment, the present invention 
is implemented within an interface unit connecting a plurality of servers to the 
Internet, which is in turn connected to a plurality of clients and on-hold server(s). 

According to a preferred embodiment of the present invention, the method 
includes the steps of opening a connection between a client and the interface unit; 
determining which server the client desires a page from; determining the current 
response time of the requested server; if the response time is acceptable then 
opening a connection between the interface unit and the requested server if no free 
connection is open between the interface xmit and the requested server; sillowing 
the client to access information on the requested server via the connections; and 
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closing the connection between the client and the interface unit while keeping open 
the connection between the interface unit and the requested server. Alternatively, 
if the response time is not acceptable, then putting the client on-hold by redirecting 
the client to an on-hold server until the response time of the requested server 
becomes acceptable. 

In another embodiment of the present invention, instead of utilizing the 
interface unit, each server has the intelligence to put the client on-hold when 
applicable. 

According to an "on-hold distribution" aspect of the invention, the interface 
unit determines the on-hold preference of the client and selects the server hosting 
that on-hold preference. 

One advantage of the present invention is that it guarantees a response time 
to a client once the client is allowed to access the requested server. 

Another advantage of the present invention is tibiat it eliminates one reason 
for server crashes, too many page requests at the same time. 

Yet another advantage of the present invention is that it temporarily puts 
a client on-hold, and once the requested server is ready to service the client, 
redirects the client back to the requested server. Putting the client on-hold both 
helps to generate additional business from the client on other servers and ensures 
that the client will be eventually redirected back to the requested server. 

Another advantage of the present invention is that clients who use the 
present on-hold invention will be given preferential access over non-clients when 
attempting to gain access to the requested server (in the case where both clients 
and non-clients of the present invention can request a web page or content from 
the requested server). 

Another advantage of the present invention is that it provides the flexibility 
to a user (via the client) to specify his or her on-hold preference through use of a 
technique referred to herein as "on-hold distribution." 
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Brief Description of the Figures 

The features and advantages of the present invention will become more 
apparent from the detailed description set forth below when taken in conjunction 
with the drawings in which like reference characters identify corresponding 
5 elements throughout and wherein: 

FIG. 1 is a network block diagram showing a plurality of clients and 
servers connected to the Internet; 

FIG. 2 is a network context diagram for an interface unit and on-hold 
server(s) according to a preferred embodiment of the present invention; 
10 FIG. 3 is a flowchart illustrating the high level operation of the present 

invention according to a preferred embodiment; 

FIG. 4 A is a detailed flowchart illustrating the on-hold aspect of the present 
invention according to a preferred embodiment; 

FIG. 4B is a more detailed flowchart illustrating the on-hold aspect of the 
15 present invention according to anotiier embodiment; 

FIG. 5 is a detailed flowchart illustrating how the present invention 
determines whether the client is read to be taken off on-hold according to a 
preferred embodiment; 

FIG. 6 is a flowchart depicting the operation of the present invention in 
20 translating client and server requests to achieve connection multiplexing; 

FIG. 7 is a message flow diagram illustrating the on-hold aspect of the 
present invention according to a preferred embodiment; 

FIG. 8 is a detailed flow diagram illustrating the translations of 
acknowledgment and sequence numbers performed by the present invention in the 
25 example of FIG. 7 according to a preferred embodiment; 

FIG. 9 depicts an example computer system in which the present invention 
can be implemented; and 
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FIG. 10 is a network context diagram where each server has the 
intelligence of the interface unit to put a client on-hold and has its own on-hold 
server according to an embodiment of the present invention. 

Detailed Description of the Preferred Embodiments 

The present invention is an apparatus, method and computer program 
product for guaranteeing network client-server response time while providing a 
way of putting the client on-hold when the response time temporarily prohibits 
access to the requested server. 

FIG. 2 is a network context diagram for an interface imit 202 and an on- 
hold server(s) 204 according to a preferred embodiment of the present invention. 
In a preferred embodiment, interface unit 202 is an intelligent network interface 
card with a CPU inside a server. Interface unit 202 can also be intelligent box 
sitting outside the server, in which case it can serve more than one server. Interface 
unit 202 can also be a load balancer, bandwidth manager, firewall, proxy-cache, 
router, switch, computer system, or any other network device that is located 
between a client and server. In theory, on-hold server 204 represents the world's 
largest web site. Users (via clients) are temporarily redirected to on-hold server 
204 when their requested server is too busy to accommodate their page requests. 
On-hold server(s) 204 may be physically located within interface unit 202, may 
be an independent server directly connected to interface unit 202 or may be an 
independent server located anywhere on the Internet. 

Referring to FIG. 2, a plurality of clients CI, C2, C3 are coupled to the 
Intemet. A plurality of servers SI, S2, S3, and on-hold server 204 are coupled to 
the Intemet by interface unit 202 (or alternatively, on-hold server 204 may be 
located anywhere on the Intemet). Servers 81, S2, S3 are collectively refen^ed to 
as a "server farm." On-hold server 204 may be a single server or a plurality of 
servers. In a preferred embodiment of the present invention, all Internet traffic 
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with the server farm passes through interface unit 202. In another embodiment, 
instead of utilizing interface unit 202, each server may have its own on-hold server 
and the intelligence to put the client on-hold when applicable, as described below 
with reference to FIG. 10. While the present invention is described in terms of the 
Internet, the concepts described also apply to other types of networks, as will be 
apparent to one skilled in the relevant art. 

Interface vmit 202 relieves servers SI, S2, S3 of much of the processing 
load caused by repeatedly opening and closing connections to clients by opening 
one or more connections with each server and maintaining these connections to 
allow repeated data accesses by clients via the Intemet. This technique is referred 
to herein as "comiection pooling." Interface unit 202 also transparently splices 
connections from servers and clients using a technique referred to herein as 
"connection multiplexing." The techniques of "connection pooling" and 
"connection multiplexing" are described in detail in related pending application 
Ser. No. 09/188,709, filed November 10, 1998, titled "Intemet Client-Server 
Multiplexer," incorporated herein by reference in its entirety. 

FIG. 3 is a flowchart illustrating the operation of the present invention witli 
respect to the network client-server guaranteed response time and on-hold aspects 
of the present invention according to a preferred embodiment. FIG. 3 incorporates 
the "connection pooling" and "connection multiplexing" teclmiques mentioned 
above. As described above, interface unit 202 may operate in either virtual IP 
and/or standard mode. When operating in virtual IP mode, the process begins 
when a client requests access to one of the servers in the server farm (herein 
referred to as the "requested server") tended by interface xmit 202. A connection 
is opened between interface unit 202 and the requesting client, and interface unit 
202 receives the client request to access the requested server, as shown in step 302. 
(Note that in standard mode, the client request would not be addressed to interface 
unit 202.) 

Interface unit 202 determines the identity of the requested server as shown 
in step 304. In one embodiment, this is accomplished by examining the 
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destination network address specified by the client request. In another 
embodiment, this is accomplished by examining the network address and path 
name specified by the client request. After determining the identity of the server 
to which the client request should be directed, interface unit 202 utiKzes the 
"connection pooling" technique by determining whether a free connection (that is, 
one that is not in use) to the requested server is already open. If not, interface unit 
202 opens a connection to the server. 

In step 306, interface imit 202 estimates the current response time of the 
requested server. The current response time of the requested server is estimated 
by interface unit 202 by using actual past response times as described next. 
However, other techniques for determining the response time can be used as 
should be readily apparent to one skilled in the art. Let t^ denote the response time 
at the i^^ episode (i.e., actual past response time). Let t/ denote the estimated 
response time at the ith episode. The response time is estimated by the following 
recxxrrence relation: 

Vd' = (((i -l)*(t u) + (ti *i))/(2i-l)) + it: ^ to * K, where K is a 
constant of error correction. 

The coefficient K is learned from the on-going traffic beginning with 
constant, and proceeding to liner, polynomial exponential changes in the error 
value. Thus the estimated response time characteristic assumes the actual response 
time. The constant is selected so as to xmder-estimate the response time 
characteristic so as to ensure the user expectations. 

There are several embodiments of this solution. One is that the time is 
measured as the time for completing an HTTP GET transaction. Another 
embodiment is to measure the time nomialized by the number of bytes transferred. 

The response time for the plurality of the servers connected to interface 
unit 202 is computed as the minimxmi of the response time of the servers behind 
it when access is through a load balancer, or on a per server basis. 
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The estimated current response time is then compared to a threshold value 
in step 308. The threshold value represents the maximum response time allowed 
for the current client (i.e., user) to gain access to the requested server. The 
threshold value is configurable by each server in the server farm, and therefore 
each server may have a different guaranteed response time. For example, a 
particular server may decide that it can service 75,000 requests with a quick 
response time, and up to 1 00,000 requests with an acceptable response time. If the 
server cannot provide the acceptable response time, then interface unit 202 is 
directed to put its clients on-hold. Thus, if the response time is greater than the 
threshold value, then control passes to step 310. If control passes to step 310, the 
client gets put on-hold. Alternatively, control passes to step 314, and the client is 
never put on-hold. 

Interface unit 202 puts the client on-hold, as shown in step 310, and as 
more fully described with respect to FIG. 4A and FIG. 4B, below. While the client 
is on-hold there is no limit to the types of information that can be provided to the 
client by the present invention, including music, sports, news and so forth. It is 
important to note that there may be one or more on-hold servers 204 utilized by the 
present invention that the client may be directed to. In the case of having multiple 
on-hold servers 204 to chose from, the present invention may use various factors 
in determining which on-hold server 204 to direct the client to. These factors may 
include such things as client IP address, type of request and on-hold server 204 
load. Hence, on-hold service may be a distributed service. An on-hold portal by 
definition should relinquish control to the request controller. This control 
switching is initiated by client-side intelligence (Java Applets/Scripts), interface 
memory (where the interface remembers the held requests) and on-hold server 204 
initiated re-direction. The re-direction can be at the IP level or at the HTTP level. 

In step 3 12, it is determined whether the client is ready to be taken oflf on- 
hold. Whether the client is ready to be taken off on-hold depends on several 
factors as described in detail with respect to FIG. 5, below. If the client is ready 
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to be taken off on-hold, then control passes to step 314. Alternatively, control 
passes to step 310 aad the client remains on-hold. 

Interface unit 202 then translates the client request and passes it to the 
requested server, as shown in step 314, and as more fully described with respect 
to FIG. 6, below. After server processing, interface unit 202 receives a response 
from the requested server, as shown in step 316. The server response is translated 
and passed to the requesting client, as shown in step 318, and described further 
below. Finally, interface unit 202 closes the connection with the client as shown 
in step 320, and the flowchart in FIG. 3 ends. However, by utilizing the 
"connection pooling" and "connection multiplexing" techniques referenced above, 
the comiection between interface unit 202 and the requested server is not 
disconnected. By maintaining open connections with the servers and by opening 
and closing connections with the client as needed, interface unit 202 frees the 
servers of nearly all of the coimection loading problems associated with serving 
clients over the Intemet. 

FIG. 4A is a flowchart depicting the operation of the present invention in 
putting the client on-hold, as shown in step 310. In step 402, interface unit 202 
determines the approximate waiting time for the client and delegates to the Java 
Applet/Script on the on-hold page. The Java Applet/Script corrects for the Roxmd 
Trip Time (due to differing network latencies). The applet displays this corrected 
waiting time which is a weighted sum of the response time given by the interface 
unit and the network latency computed by the applet. Other factors that could be 
taken into consideration when calculating the waiting time includes the estimated 
current response time of the requested server, the number of other clients already 
on-hold for the requested server and so forth. One embodiment of the present 
invention allows a web site that maintains multiple servers to provide the user (via 
the client) with the flexibility to specify his or her on-hold preference through the 
use of a "on-hold distribution" technique. This technique is described in detail 
below with reference to FIG. 4B. 
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Depending on the approximate waiting time, interface unit 202 determines 
the on-hold request to be sent to on-hold server 204, in step 404. As described 
above, on-hold server 204 in theory represents the world's largest web site. Here, 
different contents (i.e., web pages) are designed to be available based on the 
different waiting times. When the client is redirected to on-hold server 204 by 
interface imit 202, interface unit 202 determines a web page associated with an 
initial on-hold request. The initial on-hold request can be either a default request 
or a client-customized request. The initial request is a default request for all 
clients who have not been redirected to on-hold server previously by the present 
invention. Both default and customized requests return web pages containing a 
Java Applet/Script or other code (implemented by any browser programming 
language). The Java Applet/Script allows the user (via the client) to request 
additional web pages from on-hold server 204 and also to customize the user's 
initial on-hold request for use in the future. 

The customization of an initial on-hold request allows the user to configure 
the type of information (or category) to be sent to the client for display when the 
user is put on-hold by the present invention. In one embodiment of the present 
invention, the customization is achieved by returning to the user a dual 
framed/multi-window web (or HTML) page. Here, the first frame/window acts as 
a controller of the second frame/window. The second frame/window displays the 
mediimi transmitted to the user while on-hold. The first frame/window is also 
used to allow users to configure the contents of the second firame/window by 
selecting one or more categories. 

Interface unit 202 then translates the initial on-hold request and passes it 
to on-hold server 204, as shown in step 406. After request processing, interface 
unit 202 receives a response firom on-hold server 204, as shown in step 408. 
Fiaally, the on-hold server response is translated and passed to the requesting 
client, as shown in step 410, and the flowchart in FIG. 4A ends. 

FIG. 4B is a flowchart depicting the on-hold operation of the present 
invention incorporating the "on-hold distribution" technique, and thus a more 
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detail description of putting the client on-hold, as shown in step 310. In step 4 1 2, 
interface unit 202 determines the client's on-hold preference. A web site owner 
who wants to provide flexibiHty to their clients in terms of response time may use 
several servers to support tiie web site. Each server will have a different threshold 
value controlling the response time the client will experience while being serviced 
by that particxilar server. Therefore, the shorter the response time, the longer a 
client is put on-hold by the present invention. For example, a client may desire to 
have a very quick response time and thus prefer to be put on-hold until they can 
be guaranteed the quick response time. Other clients may prefer to not be put on- 
hold at all (or be put on-hold for the shortest amount of time) and thus deal with 
a slower response time. The present invention provides flexibility to the client by 
allowing a client to specify his or her preferred wait time. 

Referring to FIG. 4B, in step 412, interface unit 202 determines the client's 
(i.e., user' s) on-hold preference. Here the present invention displays options to the 
user (via the client) for being put on-hold. Based on the client's indicated on-hold 
preference, interface unit 202 determines the preferred requested server, as 
indicated by step 414. 

In step 416, interface unit 202 detemunes the approximate waiting time for 
the client and displays the approximate waiting time to the client. The 
approximate waiting time could be a factor of several variables, including the 
estimated current response time of the requested server, the number of other clients • 
already on-hold for the requested server and so forth. 

Depending on the approximate waiting time, interface unit 202 determines 
the on-hold request to be sent to on-hold server 204, in step 418. Again, different 
contents (i.e., web pages) are designed to be available based on the different 
waiting times. When the client is redirected to on-hold server 204 by interface unit 
202, interface unit 202 determines a web page associated with an initial on-hold 
request, as described above. 

Interface unit 202 then translates the initial on-hold request and passes it 
to on-hold server 204, as shown in step 420. After request processing, interface 
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unit 202 receives a response from on-hold server 204, as shown in step 422. 
Finally, the on-hold server response is translated and passed to the requesting 
client, as shown in step 424, and the flowchart in FIG. 4B ends, 

FIG. 5 is a flowchart depicting the operation of the present invention in 
determining whether the client is ready to be taken off on-hold^ as shown in step 
312, according to a preferred embodiment In step 502, interface unit 202 
estimates the current response time of the requested server, as described above 
Avith reference to FIG. 3. Note that the estimated current response time changes 
dynamically as clients are serviced. The estimated current response time is then 
compared to the threshold value in step 504. The value of the threshold in FIG. 
5 is the same as the value of the threshold in FIG. 3 . If the response time is greater 
than the threshold value, then control passes to step 509. If control passes to step 
509, the client remains on-hold. Alternatively, control passes to step 506. 

In step 506, it is determined whether the client is next to be serviced by the 
requested server. Therefore, even if the estimated current response time of a server 
indicates the server can process another request, there may be more than one client 
request to be processed. Requests can be processed in various ways, including: 
first in, first out (FIFO); assigned priorities; last in, first out (LIFO)and so forth. 
If the client is next to be serviced by the requested server, then control passes to 
step 508. Alternatively, control passes to 509, where the client remains on-hold. 

In step 508, interface unit 202 determines whether the user (via the client) 
is finished with on-hold server 204. For example, the user may want to finish with 
the page recently received from on-hold server 204. Here, if the user is not 
finished with on-hold server 204, another request may be serviced prior to the 
user's request, therefore keeping Ihe user on-hold. If the user indicates to interface 
imit 202 that the user is finished with on-hold server 204, then control passes to 
step 324 (FIG. 3), where the user is taken off on-hold. Altematively, interface unit 
202 can automatically take the user off on-hold status without any input from the 
user. Therefore, there are at least two ways to abort the on-hold service of the 
present invention including user initiated and system initiated. 
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As stated above, if the user remains on-hold for one reason or another, 
control passes to step 509. In step 509, interface unit 202 determines the 
approximate waiting time for the cUent and displays the approximate waiting time 
to the cUent. The approximate waiting time is a dynamic value which factors in 
5 the estimated current response time of the requested server, the nimiber of other 

clients already on-hold for the requested server and so forth. 

In step 510, the user (via the client) is given the opportunity to make 
another page request to on-hold server 204. Interface unit 202 then translates the 
on-hold request and passes it to on-hold server 204, as shown in step 512. After 

10 request processing, interface xmit 202 receives a response from on-hold server 204, 

as shown in step 514, Finally, the on-hold server response is translated and passed 
to the requesting client for display to the user, as shown in step 516. Control then 
passes to step 502, where the current response time is again estimated. The 
flowchart FIG. 5 continues vmtil the user is taken off on-hold. FIG. 6 is a 

15 flowchart depicting the operation of the present invention in translating client and 

server (including on-hold server 204) requests to achieve connection multiplexing, 
as shown in steps 314 and 318. In a preferred embodiment, the message traffic is 
m the form of TCP/IP packets, a protocol suite that is well-known in the art. The 
TCP/IP protocol suite supports many applications, such as Telnet, File Transfer 

20 Protocol (FTP), e-mail and Hyper-Text Transfer Protocol (HTTP). The present 

invention is described in terms of the HTTP protocol. However, the concepts of 
the present invention apply equally well to other TCP/IP applications, as will be 
apparent to one skilled in the art after reading this specification. 

Each TCP packet includes a TCP header and an IP header. The IP header 

25 includes a 32-bit source IP address and a 32-bit destination IP address. The TCP 

header includes a 1 6-bit source port number and a 1 6-bit destination port number. 
The source IP address and port number, collectively referred to as the source 
network address, uniquely identify the source interface of the packet. Likewise, 
the destination IP address and port number, collectively referred to as the 

30 destination network address, uniquely identify the destination interface for the 
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packet. The source and destination network addresses of the packet uniquely 
identify a connection. The TCP header also includes a 32-bit sequence number 
and a 32-bit acknowledgment number. 

The TCP portion of the packet is referred to as a segment. A segment 
includes a TCP header and data. The sequence number identifies the byte in the 
string of data firom the sending TCP to the receiving TCP that the first byte of data 
in the segment represents. Since every byte that is exchanged is nmnbered, the 
acknowledgment number contains the next sequence number that the sender of the 
acknowledgment expects to receive. This is therefore the sequence number plus 
one of the last successfully received bytes of data. The checksum covers the TCP 
segment, i.e., the TCP header and the TCP data. This is a mandatory field that 
must be calculated and stored by the sender and then verified by the receiver. 

In order to successfully route an inbound packet from a client to the 
intended server, or to route an outbound packet fi-om a server to a client, interface 
unit 202 employs a process known as "network address translation." Network 
address translation is well-known in the art, and is specified by request for 
comments (RFC) 1631, which can be found at the URL 
http://www.safety.net/RF70063 1 .txt. 

However, in order to seamlessly splice the client and server connections, 
the present invention also employs the novel translation technique of "connection 
multiplexing" as described in detail in related pending application Ser. No. 
09/1 88,709, filed November 1 0, 1 998, titled "Internet Client-Server Multiplexer." 
According to this technique, the present invention translates a packet by modifying 
its sequence number and acknowledgment number at the TCP protocol level. A 
significant advantage of this technique is that no application layer interaction is 
required. 

Referring to FIG. 6, the network address of the packet is translated, as 
shown in step 602. In the case of an in-bound packet (that is, a packet received 
firom a client), the source network address of the packet is changed to that of an 
output port of the interface unit and the destination network address is changed to 
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that of the intended server. In the case of an outbound packet (that is, one received 
from a server), the source netv^ork address is changed from that of the ser\^er to 
that of an output port of the interface unit and the destination address is changed 
from that of the interface unit to that of the requesting cHent. The sequence 
numbers and acknowledgment numbers of the packet are also translated, as shown 
in steps 604 and 606 and described in detail below. Finally, the packet checksum 
is recalculated to account for these translations, as shown in step 608. 

FIG. 7 is a message flow diagram illustrating the on-hold aspect of the 
present invention according to a preferred embodiment. FIG. 7 shows interface 
unit 202 connecting a client 700 to two servers, on-hold server 204 and a requested 
server 701 . For this example, it is assumed that connections have previously been 
opened v^ith on-hold server 204 and requested server 701, as described in detail 
in related pending application Ser. No. 09/188,709, filed November 10, 1998, 
titled "Internet Client-Server Multiplexer." However, it should be noted that the 
present invention can be implemented independent of the "connection pooling" 
and "connection multiplexing" techniques described in the just referenced pending 
application. 

First, interface unit 202 opens a connection with client 700 using network 
address 1 provided by client 700, as shown by flow^ 702. Once the connection is 
opened, interface unit 202 receives a GET request from client 700 specifying a 
patli name of /sales/forecast.html, as shovm by flow line 704. Assume at this 
point, interface unit 202 detennines that the GET request is for requested server 
701 and that the estimated response time for requested server 701 is such that 
client 700 must be put on-hold. 

Because a free connection is open between interface unit 202 and on-hold 
server 204, interface unit 202 does not open a ftirther connection with on-hold 
server 204. Interface unit 202 maps the appropriate GET on-hold request to on- 
hold server 204. For example, interface unit 202 can include a table that maps 
each client to that client' s customized on-hold request, if applicable, else it can use 
the default on-hold request. Interface xmit 202 forwards the GET on-hold request 
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to on-hold server 204, as shown by flow line 706. On-hold server 204 responds 
with the requested web page, as shovm by flow line 708. Interface unit 202 
forwards the web page to client 700, as shown by flow line 709, According to the 
connection pooling feature utilized by the present invention, interface unit 202 
5 does not close the connection with on-hold server 204, but rather keeps it open to 

accommodate further data flows. 

In a preferred embodiment of the present invention, the web page 
forwarded to client 700 by interface unit 202 (from on-hold server 204) contains 
two frames- One of the frames contains the Java Applet/Script which monitors the 

10 response time and computes the waiting time. The other frame displays the 

information from on-hold server 204. 

Once interface unit 202 determines that client 700 is ready to be taken off 
on-hold (as described in FIG. 5), the GET request from cUent 700 specifying a 
path name of /sales/forecast.html is sent or replayed to requested server 701, as 

15 shown by flow line 710. But, prior to this, client 700 sends a WAKE command 

to interface unit 202 and interface unit 202 responses back to client 700, as shown 
by flow line 718 and 719, respectively. Next, requested server 701 responds with 
the requested web page, as shown by flow line 712. Interface iinit 202 forwards 
the web page to client 700, as shown by flow line 714. Finally, interface unit 202 

20 closes the connection with client 700 using network address 1, as shown by flow 

line 716. 

FIG. 8 is a detailed flow diagram illustrating the translations of 
acknowledgment and sequence numbers performed by the present invention in the 
example of FIG. 7 (with the exclusion of flows 718 and 719) according to a 
25 preferred embodiment. The flows of FIG. 8 correspond to flows 702 through 716 

of FIG. 7. 

Flows 802A, B, C correspond to flow 702 and present a method of opening 
the connection between client 700 and interface unit 202. Client 700 has 
estabHshed a starting sequence nimiber of 999 and an acknowledgment number of 
30 999. Interface unit 202 responds with a SYN ACK segment specifying a starting 
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sequence nxraiber of 4999and incrementing the acknowledgment number to 1 000, 
as shown by flow 802B. Client 700 responds with an ACK segment specifying a 
sequence number of 1 000 and incrementing the acknowledgment number to 5000, 
as shown by flow 802C. Client 700 then sends a GET segment specifying a 
5 length of 50 bytes, as shown by flow 804. As in FIG. 7, assume at tliis point that 

interface unit 202 determines that the GET request is for requested server 70 1 and 
that the estimated current response time for requested server 70 1 is such that client 
700 must be put on-hold. 

Because a free open connection exists between interface unit 202 and on- 

10 hold server 204, interface tmit 202 uses that connection rather than opening a new 

connection. Interface unit 202 maps the appropriate GET on-hold request to on- 
hold server 204 by modifying the sequence and acknowledgment numbers in the 
GET segment using the sequence and acknowledgment numbers from the 
previous segment transmitted over that connection. In this example, assume that 

15 those parameters are 2000 and 7000, respectively. Also, assume that the GET 

segment for the on-hold request has a length of 30 bytes. Accordingly, interface 
unit 202 forwards the GET segment to on-hold server 204 using sequence number 
2000, acknowledgment number 7000, and length 30, as shown by flow Une 806. 
On-hold server 204 responds with the requested data specifying a sequence 

20 nimiber of 7000, an acknowledgment number of 2050, and a length of 50, as 

shown by flow 808. Interface imit 202 receives the RESP segment, translates the 
sequence and acknowledgment nimibers, and forwards the RESP segment to client 
700, as shown by flow line 810A. Client 700 responds with a RESP ACK 
segment, as shown by flow 810B. Interface unit 202 modifies the 

25 acknowledgment and sequence numbers and forwards the RESP ACK segment to 

on-hold server 204, as shown by flow 8 IOC. 

Once interface unit 202 determines that client 700 is ready to be taken off 
on-hold (as described in FIG. 5), interface uiut 202 replays the initial GET segment 
from client 700. Accordingly, because a free connection is already open with 

30 requested server 701, interface unit 202 uses that connection and translates the 



wo 01/61966 



PCT/USOl/05233 



- 19- 



sequence and acknowledgment parameters of the GET segment using the 
parameters from the previous segment transmitted over that connection. In this 
example, assume that those parameters are 3000 and 4000, respectively. 
Accordingly, interface unit 202 forwards the GET segment to requested server 
5 701 using sequence number 3000, acknowledgment nxunber 4000, and length 50, 

as shown by flow line 814. 

Requested server 701 responds with the requested data specifying a 
sequence number of 4000, an acknowledgment number of 3050, and a length of 
400, as shown by flow 816. Interface unit 202 receives the RESP segment, 

10 translates the sequence and acknowledgment numbers, and forwards the RESP 

segment to client 700, as shown by flow line 818A. Client 700 responds with a 
RESP ACK segment, as shown by flow 81 8B. Interface unit 202 modifies the 
acknowledgment and sequence numbers and forwards the RESP ACK segment to 
requested server 701, as shown by flow 818C. Finally, the connection between 

15 interface unit 202 and client 700 is closed, as shown in flows 820A, B, C, D. 

As mentioned above, another embodiment of the present invention does 
hot include interface unit 202, as described above. Here each server incorporates 
similar intelligence as described above for interface unit 202 and has its own on- 
hold server, as shown in FIG. 10. 

20 The present invention may be implemented using hardware, software or a 

combination thereof and may be implemented in a computer system or other 
processing system. In fact, in one embodiment, the invention is directed toward 
one or more computer systems capable of carrying out the functionality described 
herein. An example computer system 900 is shown in FIG. 9. The computer 

25 system 900 includes one or more processors, such as processor 904. The processor 

904 is connected to a communication bus 906. Various software embodiments are 
described in terms of this example computer system. After reading this 
description, it will become apparent to a person skilled in the relevant art how to 
implement the invention using other computer systems and/or computer 

30 architectures. 
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Computer system 900 also includes a main memory 908, preferably 
random access memory (RAM)and can also include a secondary memory 910. The 
secondary memory 1010 can include, for example, a hard disk drive 912 and/or a 
removable storage drive 914, representing a floppy disk drive, a magnetic tape 
drive, an optical disk drive, etc. The removable storage drive 914 reads from 
and/or writes to a removable storage unit 918 in a well known maimer. 
Removable storage unit 918, represents a floppy disk, magnetic tape, optical disk, 
etc. which is read by and written to by removable storage drive 914. As will be 
appreciated, the removable storage unit 918 includes a computer usable storage 
medium having stored therein computer software and/or data. 

In alternative embodiments, secondary memory 910 may include other 
similar means for allowing computer programs or other instructions to be loaded 
into computer system 900. Such means can include, for example, a removable 
storage unit 922 and an interface 920. Examples of such can include a program 
cartridge and cartridge interface (such as that foxmd in video game devices), a 
removable memory chip (such as an EPROM, or PROM) and associated socket, 
and other removable storage units 922 and interfaces 920 which allow software 
and data to be transferred from the removable storage unit 9 1 8 to computer system 
900. 

Computer system 900 can also include a communications interface 924. 
Communications interface 924 allows software and data to be transferred between 
computer system 900 and external devices. Examples of communications 
interface 924 can include a modem, a network interface (such as an Ethernet card), 
a communications port, a PCMCIA slot and card, etc. Software and data 
transferred via communications interface 924 are in the form of signals which can 
be electronic, electromagnetic, optical or oilier signals capable of being received 
by comcmimications interface 924. These signals 926 are provided to 
communications interface via a channel 928. This channel 928 carries signals 926 
and can be implemented using wire or cable, fiber optics, a phone line, a cellular 
phone link, an RF link and other communications channels. 
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In this docxmient, the terms "computer program medium" and "computer 
usable medium" are used to generally refer to media such as removable storage 
device 918, a hard disk installed in hard disk drive 912and signals 926, These 
computer program products are means for providing software to computer system 
900. 

Computer programs (also called computer control logic) are stored in main 
memory 908 and/or secondary memory 910. Computer programs can also be 
received via commimications interface 924. Such computer programs, when 
executed, enable the computer system 900 to perform the features of the present 
invention as discussed herein. In particular, the computer programs, when 
executed, enable the processor 904 to perform the features of the present 
invention. Accordingly, such computer programs represent controllers of the 
computer system 900. 

In an embodiment where the invention is implemented using software, the 
software may be stored in a computer program product and loaded into computer 
system 900 using removable storage drive 914, hard drive 912 or communications 
interface 924. The control logic (software), when executed by the processor 904, 
causes the processor 904 to perform the ftmctions of the invention as described 
herein. 

In another embodiment, the invention is implemented primarily in 
hardware using, for example, hardware components such as application specific 
integrated circuits (ASICs). Implementation of the hardware state machine so as 
to perform the ftmctions described herein will be apparent to persons skilled in the 
relevant art(s). In yet another embodiment, the invention is implemented using a 
combination of both hardware and software. 

The present invention is described specifically when implemented within 
an interface unit, such as interface unit 202, that is connected to servers in a farm 
for the purpose of offloading connection processing overhead from the servers. 
However, the present invention can also be applied within other kinds of devices 
that are in the network connection path between the client and the servers. As 
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network traffic flows through such devices, they all have the opportunity to apply 
the present invention to offload connection processing. Some examples of such 
devices are: 

- Load Balancers which distribute client network connections between a 
5 set of servers in a server farm (local or geograpMcally distributed). The invention 

can readily be combined with the load balancing function. 

- Bandwidth managers which monitor network traffic and meter packet 
flow. These devices can also use the present invention. 

- Firewalls monitor packets and allow only the authorized packets to flow 
10 through. The present invention can be used to provide an additional feature within 

firewalls. 

- Routers and switches also lie in the path of the network traffic. The 
industry trend is to integrate additional functionality (such as load balancing, 
bandwidth management and firewall functionality) within these devices. Hence, 

15 the present invention can easily be incorporated into a router. 

The specific integration of the present invention into each one of the above 
devices is implementation specific. 

The present invention can also be applied within computer systems which 
are the end points of network connections. In this case, add-on cards can be used 
20 to implement the invention and thus offload the main processing elements within 

the computer system. 
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Conclusion 

The previous description of the preferred embodiments is provided to 
enable any person skilled in the art to make or use the present invention. The 
various modifications to these embodiments will be readily apparent to those 
5 skilled in the art and the generic principles defined herein may be applied to other 

embodiments without the use of the inventive faculty. Thus, the present invention 
is not intended to be limited to the embodiments shown herein but is to be 
accorded the widest scope consistent with the principles and novel features 
disclosed herein. 
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What Is Claimed Is: 

1 . A system, comprising: 

a server, coupled to the Internet, that has stored thereon 
information, wherein a client requests said information from said server; 
5 means for estimating a response time to said clients request; 

an on-hold server, coupled to the Intemet, said server, and said 
means for determining, tliat has stored therein further information; and 

means for placing said client on-hold if said response time is more 
thaa a threshold value, whereby a connection is established between said client and 
10 said on-hold server when said client is placed on-hold. 

2, An apparatus, comprising: 

means for opening a connection between a client and an interface 

unit; 

15 means for opening a connection between said interface unit and a 

requested server if no free connection is open between said interface unit and said 
requested server; 

means for estimating a response time of said requested server; 
means for putting said client on-hold if said response time is more 
20 than a threshold value; 

means for determining when said client should be taken off on- 
hold; 

means for allowing said client to access information on said 
requested server via said connections once said client is taken off on-hold; and 
25 means for closing said connection between said client and said 

interface unit while keeping open said connection between said interface unit and 
said requested server. 
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3. The apparatus of claim 2, wherein said means for opening said 
connection between said client and said interface unit comprises: 

means for receiving a request to open a connection using a network 
address corresponding to said interface unit or said requested server; and 
5 means for receiving a request to retrieve data using a path name 

corresponding to said requested server. 

4. The apparatus of claim 3, wherein said means for opening said 
connection between said further client and said interface unit comprises: 

means for selecting said requested server as a function of said 
10 network address. 

The apparatus of claim 4, wherein said means for allowing 
means for retrieving said data from said requested server using said 

The apparatus of claim 5, wherein said means for allowing further 
means for sending said data to said client. 
The apparatus of claim 6, wherein: 

said means for receiving a request to retrieve data comprises means 
for receiving a GET segment having sequence and acknowledgment parameters; 
and 

said means for retrieving comprises 

means for modifying said parameters to produce a modified GET 

segment, and 

means for sending said modified GET segment to said requested 

server. 



5. 

comprises: 

path name. 

15 6. 

comprises: 

7. 
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8 . The apparatus of claim 2, wherein said means for putting said client 
on-hold if said response time is more than a threshold value comprises: 

means for determining a waiting time for said client; 

means for determining an on-hold request based on said determined 
waiting time; and 

means for allowing said client to access information on an on-hold 
server based on said determined on-hold request 

9. The apparatus of claim 8, wherein said means for determining said 
on-hold request comprises: 

means for allowing said client to customize said on-hold request. 

10. The apparatus of claim 9, wherein said means for allowing said 
client to customize comprises: 

means for returning to said client a dual framed/multi- window web 
page, wherein said dual firamed/multi-window web page includes a first 
frame/window and a second fi-ame/window; and 

means for allowing said client to select at least one category in said 
first firame/window thereby configuring the contents of said second fi-ame/window. 

1 1 . The apparatus of claim 2, wherein said means for determining when 
said client should be taken off on-hold comprises: 

means for recalculating said response time for said requested 

server; 

means for determining whether said client is next to be serviced by 
said requested server; 

means for determining when said client is finished with said on- 
hold server; and 
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means for taking said client off on-hold based on said means for 
recalculating, said means for determining whether said client is next to be 
serviced, and said means for determining when said client is finished. 

12. A method for guaranteeing network performance, 

5 comprising the steps of: 

opening a connection between a client and an interface unit; 
opening a connection between said interface unit and a requested 
server if no jfree coimection is open between said interface unit and said requested 
server; 

10 estimating a response time of said requested server; 

putting said client on-hold if said response time is more than a 
threshold value; 

determining when said client should be taken off on-hold; 
allowing said client to access information on said requested server 
15 via said connections once said client is taken off on-hold; and 

closing said connection between said client and said interface unit 
while keeping open said coimection between said interface unit and said requested 
server. 

13. The method of claim 12, wherein the step of opening said 
20 coimection between said client and said interface unit step comprises the steps of: 

receiving a request to open a connection using a network address 
corresponding to said interface unit or said requested server; and 

receiving arequest to retrieve data using apathname corresponding 
to said reqtiested server. 



25 



14. The method of claim 13, wherein the step of opening said 
coimection between said client and said interface unit further comprises the step 
of: 
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selecting said requested server as a function of said network 

address. 

15. The method of claim 14, wherein the step of allowing comprises 
the step of: 

retrieving said data from said requested server using said path 

name. 

16. The method of claim 15, wherein the step of allowing further 
comprises the step of: 

sending said data to said cUent. 

17. The method of claim 16, wherein: 

the step of receiving a request to retrieve data comprises the step 
of receiving a GET segment having sequence and acknowledgment parameters; 
and 

the step of retrieving comprises the steps of 

modifying said parameters to produce a modified GET segment, 

and 

sending said modified GET segment to said requested server. 

18. The method of claim 12, wherein the step of putting said cUent on- 
hold comprises of steps of: 

determining a waiting time for said client; 

determining an on-hold request based on said determined waiting 

time; and 

allowing said client to access information on an on-hold server 
based on said determined on-hold request. 
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19. The method of claim 1 8, wherein the step of putting said cUent on- 
hold further comprises the step of: 

allowing said client to customize said on-hold request. 

20. The method of claim 19, wherein the step of allowing said client 
to customize comprises the steps of: 

returning to said client a dual framed/multi-window web page, 
wherein said dual framed/multi- window web page includes a first j&ame/window 
and a second firame/window; and 

allowing said client to select at least one category in said first 
frame/window thereby configuring the contents of said second firame/window. 

2 1 . The method of claim 1 2, wherein the step of determining when said 
client should be taken off on-hold comprises the. steps of: 

recalculating said response time for said requested server; 
determining whether said client is next to be serviced by said 
requested server; 

determining when said client is finished with said on-hold server; 

and 

taking said client off on-hold based on the step of recalculating, the 
step of determining whether said client is next to be serviced, and the step of 
determining when said client is finished. 
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